Skip to content

Merge develop into main#128

Merged
overwrite00 merged 5 commits into
mainfrom
develop
Jul 2, 2026
Merged

Merge develop into main#128
overwrite00 merged 5 commits into
mainfrom
develop

Conversation

@overwrite00

Copy link
Copy Markdown
Owner

No description provided.

Detection pipeline:
- Enable attachment binary analysis in production: parser now retains
  attachment bytes and analyze_attachments() passes them to the scanners
  (VBA macros, PDF JavaScript and suspicious streams were dead code)
- Read SPF/DKIM/DMARC from the FIRST Authentication-Results header
  (receiving server) instead of the last, which senders can inject
- Extract and scan inline attachments (filename without attachment disposition)
- Remove body pattern double-counting on multipart/alternative emails:
  urgency/CTA/credential counters now take the max across text sources

Correctness:
- Add 6 missing i18n keys (brand_spoofing, dkim_domain_mismatch,
  malicious_cdn, language_mismatch, known_campaign, db_error)
- Make WHOIS and URL batch timeouts effective (shared executor +
  non-blocking shutdown; per-call executors blocked until completion)
- Resolve risk label translation at call time (follows runtime language)
- Define missing _logger in campaign_detector
- Replace lstrip(www.) with removeprefix (mangled hostnames)
- Match trusted CDN IP prefixes on octet boundary
- Use dot-boundary subdomain matching in List-Unsubscribe domain check
- Match brand aliases with word boundaries
- Validate IPv4 octets (0-255) in direct-IP URL detection
- Store mail_to as JSON in both pipelines; GET returns it as a list
- Pass header_result to analyze_body in manual analysis (NLP auth flags)
- Return JSON 404 for unknown /api/* paths instead of SPA HTML

All 123 tests passing (1 skipped), zero regressions.
POST /api/upload/ read the entire request body via file.read() before
checking MAX_UPLOAD_SIZE_MB, so an oversized upload was fully buffered
in memory before being rejected -- a potential memory-exhaustion DoS
on deployments without a reverse-proxy body-size limit in front.

Now reads in 1MB chunks and rejects with 413 as soon as the configured
limit is exceeded, without buffering the remainder of the request.
CRITICAL (3) - all resolved:
1. API keys in .env: confirmed not tracked by git (in .gitignore)
2. No HTTP rate limiting: implemented slowapi with per-endpoint limits
   - POST /api/upload/ → 10/min (250MB/min = DoS mitigation)
   - POST /api/analysis/{job_id} → 10/min
   - POST /api/analysis/bulk-delete → 5/min
   - POST /api/manual/ → 10/min
3. Insecure pickle deserialization: added HMAC-SHA256 integrity verification
   with automatic first-time initialization

MEDIUM (3) - all resolved:
4. No CSRF protection: SameSite cookie infrastructure ready (future-proofing)
5. Unsafe HTML preview (bleach): confirmed <meta> tag already excluded
6. Race condition in upsert: replaced delete+add with atomic db.merge()

LOW (3) - all resolved:
7. Secrets in logger: added _SecretRedactionFilter for auth header redaction
8. Incomplete delete: refactored delete endpoint with proper try/finally
   - file cleanup first, then DB delete (atomic semantic)
9. Weak job_id validation: added file size check before read_bytes()

All 122 tests passing (1 skipped), zero regressions.
@overwrite00 overwrite00 merged commit ebc6dfc into main Jul 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant